Skip to content

test(system_test): add the catalog_build SVT theme - #5795

Open
Nagachandan-P wants to merge 6 commits into
dell:stagingfrom
Nagachandan-P:svt-catalog-build
Open

Nagachandan-P wants to merge 6 commits into
dell:stagingfrom
Nagachandan-P:svt-catalog-build

Conversation

@Nagachandan-P

Copy link
Copy Markdown
Collaborator

Description of the Solution

Adds a new SVT theme, catalog_build, to the system test framework from #5711. It runs Omnia catalogs one after another on an OIM: for each catalog it s
elects the catalog (./omnia.sh --select-catalog), then runs the Repo Manager and Image Build Manager FVT steps (exec, then verify). The result is o
ne SVT report with one stage per catalog. The theme is defined as data in the same way as build_stream, and it uses the shared runner, readiness check
s, --retry / --start-stage and the HTML/JSON report, so it doesn't add a separate execution loop.

Related Issue

Changes

Theme definition (test/system_test/library/vars/)

  • catalog_vars.py (new): reads the catalog plan written at configure time and builds one stage per catalog (catalog_01, catalog_02, ...), its title
    and its test cases, with IDs SVT_CATALOG_BUILD_<NN><SS>.
  • theme_vars.py: registers the catalog_build theme (stages, soft stages, credential domains, cleanup and report settings), using the same fields as build_stream. force_rebuild defaults to true.
  • test_case_vars.py: adds the generated catalog test cases to TEST_CASES.

Catalog runtime (test/system_test/library/functions/)

  • catalog_settings.py (new): validates the catalog_build: block (catalog list, include_aarch64, repo_config, caching_policy, repo_policies, force_rebuild, continue_on_failure, cleanup_between, steps).
  • catalog_func.py (new):
    • resolves catalogs: all or an ordered list against ./omnia.sh --list-catalogs, and skips aarch64 catalogs (with the reason) unless include_aarch6 4: true;
    • writes and checks reports/catalog_build_plan.json;
    • before each catalog, writes the run-wide repo_manager and image_build_manager settings into the runtime inputs, then selects the catalog;
    • runs cleanup_between (none, images, repos or full);
    • after a failed or timed-out step, stops orphaned ansible-playbook <domain>.yml workers and their child processes.

Shared framework hooks (opt-in; build_stream tests don't use them)

  • test_func.py: optional test-case fields:
    • action: run an SVT action (catalog select or skip) instead of an FVT step;
    • always: run the test even after an earlier failure, used by cleanup_between;
    • stop_leftovers: clean up leftover playbook processes after a failed exec.
  • fvt_func.py: SVT actions show as svt <action> ... in the report.
  • config_func.py: loads and validates the catalog_build: block for the catalog theme.
  • readiness_func.py: adds the initialization check "Catalog plan of this run (configure)".
  • setup_func.py: makes bundled_catalogs() and select_catalog() public.
  • _setup.py: --configure resolves the catalog plan and prints it; a plan error fails configure.
  • svt_msgs.py: new CATALOG_MSGS.

Configuration and inputs

  • catalog_build_test_config.yml (new): theme settings; ships with catalogs: all and gives an ordered list as a commented example.
  • dataset_generator/profiles/catalog_build.yml (new): input profile for omnia.env, repo_manager and image_build_manager.

Documentation and tests

  • docs/CATALOG_BUILD.md (new): user guide covering prerequisites, quick start, settings reference, reading the report, retry/resume and troubleshootin
    g.
  • catalog_build_design.md (new): design of the theme on the merged SVT framework.
  • README.md: catalog_build section and link to the guide.
  • ut/test_catalog_build.py (new): unit tests for settings, the plan, generated stages and tests, readiness, the select/skip/cleanup actions, leftover-p
    rocess cleanup and the run_svt_test hooks.

Files Changed

File Change Type Description
test/system_test/catalog_build_test_config.yml Added Theme settings (catalogs: all, run-wide repo/image settings, steps, cleanup, report)
test/system_test/catalog_build_design.md Added Design document for the theme
test/system_test/docs/CATALOG_BUILD.md Added User guide
test/system_test/dataset_generator/profiles/catalog_build.yml Added Dataset generator profile (omnia.env, repo_manager, image_build_manager)
test/system_test/library/vars/catalog_vars.py Added Generates stages, stage titles and test cases from the catalog plan
test/system_test/library/functions/catalog_settings.py Added Validation of the catalog_build: block
test/system_test/library/functions/catalog_func.py Added Catalog plan, select/skip, run-wide settings, cleanup_between, leftover-process cleanup

|
| test/system_test/ut/test_catalog_build.py | Added | Unit tests for the theme |
| test/system_test/library/vars/theme_vars.py | Modified | Registers the catalog_build theme |
| test/system_test/library/vars/test_case_vars.py | Modified | Adds the generated catalog test cases |
| test/system_test/library/functions/config_func.py | Modified | Loads the catalog settings for the catalog theme |
| test/system_test/library/functions/readiness_func.py | Modified | Catalog plan readiness check |
| test/system_test/library/functions/setup_func.py | Modified | Makes the catalog list/select helpers public |
| test/system_test/library/functions/test_func.py | Modified | Optional action / always / stop_leftovers test-case fields |
| test/system_test/library/functions/fvt_func.py | Modified | Report label for SVT actions |
| test/system_test/library/messages/svt_msgs.py | Modified | Catalog messages |
| test/system_test/_setup.py | Modified | Configure writes and prints the catalog plan |
| test/system_test/README.md | Modified | catalog_build section and guide link |

Testing

  • Unit tests: .venv/bin/python -m pytest ut -q gives 40 passed.

  • Static checks:

    • pylint scores 10/10 on the new and changed code. The only warning is a too-many-locals warning in setup_func.py that was already there.
    • bandit reports no issues.
  • build_stream regression: test collection is unchanged (13 build_stream tests, same stages and soft stages).

  • End-to-end run on an OIM (RHEL 10.2, x86_64, run svt-20261112T061055Z), with an explicit two-catalog list:

    Stage Catalog Result Duration
    initialization - Passed (7 checks) 0s
    catalog_01 rhel/10.2/slurm_x86_64_no_vast.json 17/17 passed 35m16s
    catalog_02 rhel/10.2/slurm_service_k8s_x86_64_no_vast.json 17/17 passed ~42m
    summary - Passed 1s

    Settings were repo_config: partial, caching_policy: true, force_rebuild: true, continue_on_failure: true and cleanup_between: none. run_vali dation.sh svt_catalog_build all exited 0, and the HTML/JSON report was generated with OVERALL STATUS: PASSED.

  • Commands used:

    ./setup_env.sh --theme catalog_build
    ./setup_env.sh --theme catalog_build --status
    ./run_validation.sh svt_catalog_build list
    ./run_validation.sh svt_catalog_build all
    
    

• Not run:
• catalogs: all (every bundled catalog). It takes many hours; plan resolution and ordering for all are covered by unit tests.
• aarch64 catalogs ( include_aarch64: true ). No aarch64 build host was available.
• The GitLab pipeline generator for this theme isn't covered: it needs a configured run's plan file.

Backward Compatibility

• No breaking changes.
• The build_stream theme behaves exactly as before:
• the new test-case fields ( action , always , stop_leftovers ) are optional, and build_stream test cases don't set them;
• the catalog plan check runs only for catalog_build ;
• bundled_catalogs() / select_catalog() were only made public, with no change in behavior.
• No changes to user_inputs.yml , the src/ domain inputs or any FVT suite. The theme reuses the existing test/repo_manager and test/image_build_manag
er FVT runners.
• No migration needed. The new theme is opt-in via ./setup_env.sh --theme catalog_build .

Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com>
Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com>
Signed-off-by: Nagachandan P <Nagachandan.p@dell.com>
Signed-off-by: Nagachandan-P <Nagachandan.p@dell.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant